Comparison of intuitive assessment and palliative care screening tool in the early identification of patients needing palliative care

The intuitive assessment of palliative care (PC) needs and Palliative Care Screening Tool (PCST) are the assessment tools used in the early detection of patients requiring PC. However, the comparison of their prognostic accuracies has not been extensively studied. This cohort study aimed to compare the validity of intuitive assessment and PCST in terms of recognizing patients nearing end-of-life (EOL) and those appropriate for PC. All adult patients admitted to Taipei City Hospital from 2016 through 2019 were included in this prospective study. We used both the intuitive assessment of PC and PCST to predict patients’ 6-month mortality and identified those appropriate for PC. The c-statistic value was calculated to indicate the predictive accuracies of the intuition and PCST. Of 111,483 patients, 4.5% needed PC by the healthcare workers’ intuitive assessment, and 6.7% had a PCST score ≥ 4. After controlling for other covariates, a positive response ‘yes’ to intuitive assessment of PC needs [adjusted odds ratio (AOR) = 9.89; 95% confidence interval (CI) 914–10.71] and a PCST score ≥ 4 (AOR = 6.59; 95%CI 6.17–7.00) were the independent predictors of 6-month mortality. Kappa statistics showed moderate concordance between intuitive assessment and PCST in predicting patients' 6-month mortality (k = 0.49). The c-statistic values of the PCST at recognizing patients’ 6-month mortality was significantly higher than intuition (0.723 vs. 0.679; p < 0.001). As early identification of patients in need of PC could improve the quality of EOL care, our results suggest that it is imperative to screen patients’ palliative needs by using a highly accurate screening tool of PCST.

Palliative care (PC) services markedly improve the quality of end-of-life (EOL) care in terminally ill patients 1,2 . The World Health Organization (WHO) has estimated that 40 million individuals with cancer and other lifelimiting diseases need PC, however only 14% receive it 3 . According to the WHO, PC should be initiated in an early phase of life-limiting diseases and not be restricted to terminal care or the last six to twelve months of life 4 . However, early identification of both patients nearing EOL and those appropriate for PC is a challenge for healthcare systems. Prior reports have shown that clinicians are inaccurate at prognostication and identification of dying patients 5,6 .
Intuitive assessment of PC needs and Palliative Care Screening Tool (PCST) are the tools that are necessary to identify patients nearing the EOL and provide palliative care service for those who need this care [7][8][9] . An intuitive assessment of PC needs comprises asking the healthcare workers whether a patient will need PC services or whether the patient is going to die within 6-12 months 9 . Since the intuitive assessment of PC needs does not require clinicians to collect patients' clinical data or use a complex scoring algorithm, it has been widely used in www.nature.com/scientificreports/ assisting healthcare practitioners in recognizing patients nearing EOL and those appropriate for PC 9 . However, the accuracy of intuitive assessment varies by study population, ranging from a poor to a reasonable accuracy 9 . The PCST is another assessment method assisting healthcare workers in the early identification of patients nearing EOL and in need of PC 7,8,10 . PCST collects patients' clinical data (e.g. comorbidities) and uses a scoring algorithm to estimate their period of survival 8,10 . Although intuitive assessment of PC and PCST are the assessment tools assisting healthcare providers in identifying patients appropriate for PC, the comparison of their prognostic accuracies remains unexplored. A previous Australian cohort study followed 4365 patients aged 70 years and over to compare the validity of intuition and the Supportive and Palliative Care Indicators Tool (SPICT) in predicting their mortality 11 . This study showed that the SPICT had a higher sensitivity but a lower specificity in predicting patients' 12-month mortality, compared to intuition 11 .
Determining the validity of intuitive assessment and PCST for early identification of patients nearing EOL could inform future PC policies and practices to guide accurate screening for patients' needs and improve the scope of PC services. The goal of this cohort study was to compare the prognostic accuracy of intuitive assessment and PCST in the early identification of patients needing PC.

Methods
Background information and study participants. Taipei City Hospital (TCH), a 4700-bed facility, is the largest healthcare organisation in Northern Taiwan. TCH, since 2015, has initiated a large-scale palliative care program to facilitate early identification of patients in need of PC and to provide EOL discussions for the patients 12 . When patients are admitted to TCH, their PC needs were evaluated by the primary care nurses using the PCST (as shown in Supplementary Table 1) 13 and intuitive assessment of PC needs. The PCST is a one-page screening tool adapted from the instrument used in St. Mary's medical center and Center to Advance Palliative Care in the United State 13 . The total score of PCST ranges from 0 to 31. If patients' PCST score is 4 points or greater, they are informed about EOL discussions to discuss their preference regarding EOL care. To promote EOL discussions to patients nearing death, EOL discussions are also initiated for patients whose PC needs are ascertained through intuitive assessment by healthcare workers. During the EOL discussions, healthcare providers discuss patients' goals and preferences concerning medical care towards the end of their lives.
This study included participants who were aged 18 years or older and were admitted to TCH from 2016 through 2019. All patients were followed up until December 31, 2020 or their death. The Institutional Review Board of TCH (no. TCHIRB-11003001-E) approved this study. The informed consents for study participants were waived in this report. All methods in this study were performed in accordance with relevant guidelines and regulations. This study is also in accordance with the Declaration of Helsinki.
Palliative care training program for healthcare providers. TCH, since 2015, has held a series of 26 palliative training programs for all healthcare providers in order to early identify patients nearing the EOL and provide PC services to those who need them 14  Outcome variable. The primary outcome of this study was the 6-month mortality risk of patients. Deaths were determined by patients' medical records.
Main explanatory variable. The main explanatory variables were the intuitive assessment of PC needs and PCST score. When patients are admitted to TCH, their primary care nurses used intuitive assessment to evaluate their palliative needs through the question, 'Would I agree that this patient needs palliative care services?' The responses to the intuitive assessment of PC needs included either 'yes' or 'no' . PCST scores were categorised into < 4 or ≥ 4 points. Our prior validity study found that the specificity of a PCST score ≥ 4 in predicting 6-months mortality was 91.8% among hospitalised patients 13 .
Controlling variables. The controlling variables included sociodemographics, hospital units, and comorbidities. The sociodemographic characteristics included age and gender. Hospital units were categorized as general ward and intensive care unit. Comorbidity was determined using the patients' medical records. The comorbidities included cancer, heart failure, chronic obstructive pulmonary disease (COPD), liver cirrhosis, end-stage renal disease, and cerebrovascular accident.
Statistical analysis. First, we analyzed the subjects' demographic data. We then presented continuous data as mean ± standard deviation (SD), and conducted a two-sample t test to compare outcomes between patients who needed PC services and those who did not. We analyzed categorical data using the Pearson χ 2 test where appropriate.
We used Kaplan-Meier method to generate survival curves, with comparisons being evaluated according to the intuitive assessment of PC needs and PCST scores of patients. Multivariate logistic regression was used to estimate the association of intuitive assessment and PCST scores with 6-month mortality after adjusting for www.nature.com/scientificreports/ participants' age, gender, and comorbidities. The variable with p < 0.05 was defined as a significant factor associated with the outcomes in the multivariate analysis. Adjusted odds ratios (AOR) with 95% confidence intervals (CI) were reported to show the strength and direction of these associations. We used 2 × 2 contingency tables to calculate the concordance between intuitive assessment and PCST scores in predicting patients' 6-month mortality 15 . The strength of this agreement was assessed using the kappa (k) statistic.
To assess the prognostic prediction of intuitive assessment and PCST scores, we calculated the specificity (the ability to recognize those not nearing the EOL), sensitivity (the ability to recognize patients nearing the EOL), positive predictive value (PPV; the proportion of patients who died when the healthcare providers predicted nearing the EOL), and the negative predictive value (NPV; the proportion of patients who survived when the healthcare providers predicted survival) 16 . We estimate the c-statistic value, also known as the area under the curve, to indicate the level of predictive accuracy of intuition and PCST 17 . A score of 0.5 suggests a model with poor predictive value, meaning that healthcare providers are no better than chance at identifying a patient nearing EOL. An increase in the c-statistic value indicates an increase in the level of predictive accuracy. A good predictive model requires the c-statistic score to be > 0.7. The difference of c-statistic value for intuitive assessment and PCST was compared from different logistic regression models 18 . All data management and analyses in this study were performed using the SAS 9.4 (SAS Institute, Cary, NC) and STATA 13.0 (STATA Corp, College Station, TX) software package.

Results
Participant selection. This cohort study included 130,361 patients who were admitted to TCH and were evaluated for PC needs, from 2016 through 2019. After excluding those younger than 18 years (n = 18,878), the remaining 111,483 patients were included in the analysis. The overall mean (SD) age was 60.9 (19.1) years; and 50.6% of the subjects were male. Of all study subjects, 4,984 (4.5%) individuals needed PC services based on intuitive assessment by healthcare workers, and 7428 (6.66%) subjects obtained a PCST score ≥ 4. Table 1 shows the characteristics of patients according to intuitive assessment of PC needs. Compared with patients without PC needs, those with PC needs were older (76.3 vs 60.2 years). Moreover, patients with PC needs showed www.nature.com/scientificreports/ higher proportions of comorbidities and were more likely to be admitted to the intensive care unit. The proportion of patients with PCST scores ≥ 4 were 64.29% and 3.97% in those with and without PC needs, respectively. Additionally, 45.12% of patients with PC needs had undergone EOL discussions with physicians, with only 5.07% of patients without PC needs receiving the same discussions.

Mortality rates by intuitive assessment of palliative care needs and PCST.
During the follow-up period, 3,978 patients died within 6 months of palliative care screening, including 1549 (31.08%) individuals with PC needs and 2429 (2.28%) patients without PC needs. The proportion of 6-month mortalities were 26.56% and 1.92% in patients with PCST scores of ≥ 4 and < 4, respectively. Time to death was significantly shorter in patients with PC needs when compared to those without PC needs (P < 0.001, log-rank test; Fig. 1A). When compared with patients with PCST scores of < 4, the mortality rate was significantly higher among those with scores ≥ 4 (P < 0.001, log-rank test; Fig. 1B).
Factors associated with 6-month mortality among patients who received palliative care screening. Multivariate logistic regression was used to identify the independent risk factors for 6-month mortality in patients assessed for PC needs. After controlling for demographics and comorbidities, patients with PC needs showed a significantly higher risk of 6-month mortality compared to those without PC needs (AOR = 9.89; 95% CI 9.14-10.71; P < 0.001) ( Table 2). Moreover, a PCST score ≥ 4 was associated with a higher risk of 6-month mortality (AOR = 6.59; 95% CI 6.17-7.00; p < 0.001). The independent risk factors for 6-month mortality were age ≥ 65 years, being male, admission to intensive care unit, cancer, heart failure, COPD, liver cirrhosis, and cerebrovascular accident.

Correlation between intuitive assessment and PCST in predicting patients' 6-month mortality.
Kappa statistics showed moderate concordance (k = 0.49) between intuitive assessment and PCST in predicting patients' 6-month mortality (Table 3). Patients with both PCST score ≥ 4 and PC needs reported as 'no' were major contributors to the discrepancy between intuitive assessment of PC needs and PCST in predicting patients' 6-month mortality. Table 4 shows the accuracy of the intuitive assessment of PC needs and PCST at predicting 6-month mortality in patients. The c-statistic values of the intuitive assessment of PC needs and PCST at recognizing patients in last 6 months of life were 0.679 and 0.723, respectively (p < 0.001).

Discussion
In this cohort study of 111,483 patients, 3978 (3.57%) individuals died within 6 months of palliative care screening. After adjusting for demographics and comorbidities, a positive response 'yes' to intuitive assessment of PC needs and a PCST score ≥ 4 were found to be the independent predictors for patients' 6-month mortality. Additionally, Kappa statistics showed moderate concordance between intuitive assessment and PCST in predicting patients' 6-month mortality. A comparison of prognostic accuracy of intuitive assessment and PCST indicated that PCST was better at predicting 6-month mortality in patients than intuitive assessment. Early identification of patients nearing EOL is an important step to deliver PC services to those who need such care 19 . Intuitive assessment and PCST are the screening tools assisting healthcare providers in predicting patient outcomes and early identification of those in need of PC 7,9 . However, the comparison of the validity of intuitive assessment and PCST has not been extensively studied. An Australian study involving 4365 patients found that PCST was better at predicting 12-month mortality than intuitive assessment (1.6% vs. 1.1%) although with no significant difference between these two tools 11 . Our study followed 111,483 patients and found that the prognostic accuracy of PCST was significantly better than intuitive assessment (72.3% vs. 67.9%). As early identification of people nearing EOL could create greater opportunities for providing PC services, as well as improve their quality of EOL care 1 , our study suggests that it is important to screen patients' palliative needs via an accurate screening tool.
This study found that 33.6% of deceased patients underwent EOL discussions with physicians before their death, which was higher than 31.2% among deceased cancer patients in the US 20 . The palliative program to promote EOL discussions at Taipei City Hospital may account for the high proportion of Taiwanese patients undergoing EOL discussions nearing the end of their lives. The Taipei City Hospital, since 2015, has initiated a large-scale PC program to early identify patients nearing the EOL 12 . A palliative care screening tool was embedded into the existing electronic medical healthcare system to evaluate patients' PC needs. The duration of completing an electronic PCST assessment for each patient was about five minutes. Patients with a PCST score ≥ 4 or a positive response 'yes' to an intuitive assessment of PC needs were informed about EOL discussions to determine their preference regarding EOL care. During the EOL discussions with physicians, patients' expectations and goals regarding the EOL care were discussed and emphasized. Since EOL discussions improved patient quality of care 21,22 , the findings of our study suggest that it is important to incorporate a comprehensive palliative care screening tool into the existing healthcare system to early identify hospitalized patients nearing EOL and provide EOL discussions to better align care with their preferences.
Palliative programs for early identification of patients nearing EOL are less common in the regions of Asia 23 . Taiwan has been a leader in Asia with regard to palliative care 24 and launched the first Patient Self-Determination Act (PSDA) in 2015 25 . Since a sustainable palliative program service should include all health care professionals across different specialty who are able to provide a palliative care approach for patients with serious illnesses 26 www.nature.com/scientificreports/ providers in the provision of PC and EOL discussions for patients 14 . Although integrated PC programs have been seldom implemented in Asia 28 , our study demonstrated that it is feasible to implement a comprehensive palliative program for early identification of patients needing PC and promote EOL discussions for the patients. www.nature.com/scientificreports/ There are two strengths in our study. First, our cohort study was a large-scale study comparing the validity of intuitive assessment and PCST to recognize patients nearing the EOL. Our study found that the accuracy of PCST was better than intuitive assessment in predicting patients' 6-month mortality. Since early identification of patients nearing EOL could facilitate more effective palliative care planning, our study suggests that early identification using an accurate screening tool is imperative. Second, although palliative programs for early identification Table 2. Univariate and multivariate analyses of factors associated with 6-month mortality among patients. *** < .001. AOR, adjusted odds ratio; CI, confident interval; COPD, chronic obstructive pulmonary disease.

Variables
Number of patients  www.nature.com/scientificreports/ of patients in need of PC are uncommon in Asia 23 , our study suggests that it is practical to incorporate a palliative screening program into the existing healthcare system to ensure early identification of patients in need of PC. Nonetheless, one limitation should be considered in interpreting our findings. The external validity of our findings may be a concern because almost all our enrolees were Taiwanese. The generalizability of our results to other, non-Asian ethnic groups thus requires further verification.
In summary, this population-based cohort study found that an intuitive assessment of PC needs with a positive response 'yes' and a PCST score ≥ 4 were the independent predictors for 6-month mortality in patients. There was a moderate concordance between intuitive assessment and PCST in predicting patients' 6-month mortality. In terms of prognostic accuracy, PCST was better than intuitive assessment in predicting patients' 6-month mortality. As the early identification of patients in need of PC could help in effectively meeting their treatment goals and improve the quality of EOL care, it is imperative to screen patients' palliative needs by using a highly accurate screening tool.